WO 2005/116938 



-30- 



PCT/JP2005/010202 



CLAIMS 

1. An image processing method of presenting a 
virtual object superimposed on a physical space to an 
observer, characterized by comprising: 
5 a holding step of holding, in a memory, 

information of a state of each virtual object included 
in a virtual space; 

an association step of associating at least one 
virtual object included in the virtual space with at 
10 least one position/orientation sensor which is held in 
a hand of the observer and manipulated; 

a layout step of laying out, in the virtual 
space, the virtual object associated with the 
position/orientation sensor in the association step in 
15 accordance with a position and orientation of the 
position/orientation sensor itself, which are 
determined on the basis of a measurement result by the 
posit ion/ orientation sensor; 

a presentation step of presenting the state of 
20 each virtual object included in the virtual space to 
the observer on the basis of the information held in 
the holding step; 

an input step of inputting an operation to change 
a desired one of the states presented in the 
25 presentation step; and 

an updating step of updating the information in 
accordance with the operation input in the input step. 
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2. The method according to claim 1, characterized by 
further comprising a registration step of registering, 
in the information, the position and orientation of the 
5 position/orientation sensor itself, which are 

determined on the basis of the measurement result by 
the position/orientation sensor as a position and 
orientation of the virtual object associated with the 
position/orientation sensor in the association step, 
10 wherein in the layout step, the virtual object is 

laid out in the virtual space by using the position and 
orientation of the virtual object registered in the 
information in the registration step. 



15 3. The method according to claim 1 or 2 , 

characterized in that the information includes a name, 
blinking state, selected/unselected state, and section 
display state of the virtual object. 



20 4. An image processing method including 

acquiring position and orientation information of 
an observer, 

acquiring an image of a physical space; 
generating an image of a virtual space in 
25 accordance with the position and orientation 
information of the observer, and 

compositing the image of the physical space and 
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the Image of the virtual space and displaying the 
composited image on a head mounted display worn by a 
user, characterized by comprising: 

generating an operation panel image and 
5 compositing the operation panel image with the image of 
the physical space and the image of the virtual space; 

acquiring position information of an operation 
unit operated by the observer; and 

updating the operation panel image in accordance 
10 with a positional relationship between the operation 
panel image and the operation unit , 

wherein in the operation panel image, a part 
selected by the operation unit is enlarged. 

15 5. The method according to claim 4, characterized in 
that 

the image of the virtual space is generated on 
the basis of a 3D CAD data of a virtual object, 

the operation panel contains an assembly tree 
20 based on the 3D CAD data, and 

the enlarged part includes a component name 
contained in the assembly tree. 



6. An image processing apparatus for presenting a 
25 virtual object superimposed on a physical space to an 
observer, characterized by comprising: 

holding unit adapted to hold information of a 
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state of each virtual object included in a virtual 
space ; 

association unit adapted to associate at least 
one virtual object included in the virtual space with 
5 at least one position/orientation sensor manipulated by 
the observer; 

layout unit adapted to lay out, in the virtual 
space, the virtual object associated with the 
position/orientation sensor by said association unit in 
10 accordance with a position and orientation of the 
position/orientation sensor itself, which are 
determined on the basis of a measurement result by the 
position/orientation sensor; 

presentation unit adapted to present the state of 
15 each virtual object included in the virtual space to 
the observer on the basis of the information held by 
said holding unit; 

input unit adapted to input an operation to 
change a desired one of the states presented by said 
20 presentation unit; and 

updating unit adapted to update the information 
in accordance with the operation input by said input 
unit . 

25 7. A program characterized by causing a computer to 
execute an image processing method of any one of claims 
1 to 5. 
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8. A computer-readable storage medium characterized 
by storing a program of claim 7 . 



